Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation

Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes handling flexible rates (e.g., increasing rate more dynamic portion video as well missing frames). To resolve restricted nature existing models' ability handle arbitrary timesteps, we propose continuous-time by combining neural ODE (Vid-ODE) with pixel-level processing techniques. Using ODE-ConvGRU an encoder, a convolutional version recently proposed ODE, enables us learn dynamics, Vid-ODE can spatio-temporal dynamics input videos rates. The decoder integrates learned function synthesize frames at any given where composition technique is used maintain sharpness individual frames. With extensive experiments on four real-world datasets, verify that outperforms state-of-the-art approaches various settings, both within trained time range (interpolation) and beyond (extrapolation). best our knowledge, first work successfully performing using videos.